1. Field
The present invention relates to a system and method for controlling an autonomous mower, e.g. lawn mower, and for adapting its perception, i.e. the combined sensing of the mower's environment (“environment” being the input field of the sensor(s)). In particular the present invention is for performing a combined control of sensor parameters, motor commands, and camera image computations of the autonomous mower. The present invention further relates to an autonomous mower equipped with said control system.
2. Description of Related Art
Autonomous or robotic mowers, e.g. lawn mowers, are an increasing market. Such mowers typically mow a lawn autonomously in a random brute-force fashion cutting only small pieces of grass in every run. Small grass pieces fall into the sod, thereby fertilizing the lawn. This principle is called mulching.
The term “autonomous mower” is well known to those skilled in the art an refers to an unmanned mower which has an autonomous drive unit in order to move the mower (“self-driving mower”), an onboard energy reservoir to power the drive unit, at least one sensor and a computing unit functionally connected to the sensor(s) and the drive unit.
In many commercial approaches of autonomous lawn mowers the mowing area is delimited by an electric border wire, which emits a weak electromagnetic field. This field is used by the autonomous mower to stay within the allowed mowing area, and to find a base station for recharging.
For avoiding static or dynamic obstacles that are not indicated by the border wire, some commercially available autonomous mowers use bump and sonar sensors. However, such mowers will still drive over small obstacles (like e.g. cellular phones, hoses or garden tools) lying on the grass, since such small obstacles are neither indicated by the border wire nor detected by the bump or sonar sensor. On the one hand side this can cause severe damage to the mowing blades, and on the other hand side this can also damage the small obstacles. Furthermore, most available autonomous mowers bump into objects, before they turn away, even those mowers that additionally use sonar sensors. This ultimately leads to many scratches on the outer shell of the mowers.
Some theoretical and/or experimental approaches are known in the state of the art, which seek to improve the commercially available mowers, wherein in particular the border wire is to be removed.
EP 2 286 653 A2 describes an autonomous lawn mower that combines a camera recognition with the output of a grass sensor. The grass sensor is used to bootstrap and update a classifier that classifies pixels as grass or non-grass based on their pixel values (e.g. color). Furthermore, the claims of the application describe a computer implementation of controlling a mower with a camera. However, the technique is limited to certain operation limits of the camera sensor. An intelligent control of the camera is not performed.
DE 103 27 223 A1 describes a mower, which uses laser light to scan the ground for non-grass areas, which are then avoided. Furthermore, the document describes a mower with a camera and a light unit with norm light for spectral analysis. However, the patent application is limited to good lighting conditions.
US 2006/0151680 A1 describes an active illumination on a lawn mower for recognizing the condition of turf grass. The document encompasses an adaptation of an active lighting, a layout of lighting (concentric circles) and a kind of lighting (LED).
U.S. Pat. No. 6,832,000 B2 suggests grass segmentation by calculating a color and a texture probability for each pixel and by classifying each pixel as grass or non-grass based on the probabilities. The patent, however, does not intend to apply the technique to autonomous lawn mowing. In particular, the described processing is not used to control movement of an autonomously mowing device.
“Autonomous configuration of parameters in robotic digital cameras, Neves et al, Proc. of the 4th Iberian Conference on Pattern Recognition and Image Analysis, ibPRIA 2009, Póvoa do Varzim, Portugal (June 2009)” explains a combined exposure, gain, brightness and white-balance control. However, no scene specific adaptation is carried out. The adaptation is based on the whole image only.
“Automatic Camera Exposure Control, N. Nourani-Vatani and J. Roberts, Australasian Conference on Robotics and Automation (December 2007)” describes an automatic exposure control for an omni-camera, which also features a mask for excluding the dark parts of a mirror support, which would lead to a wrong control due to bias. The mask can also be non-binary, which allows for a weighted influence of each pixel. However, the paper does not target for an optimal camera control for lawn mowers.
“Identification & Segmentation of Lawn Grass Based on Color and Visual Texture Classifiers, A. Schepelmann, Case Western Reserve University (CWRU), Master Thesis (August 2010)” describes a grass recognition system for an autonomous lawn mower (CWRU cutter). The grass detection is based on color and texture cues and the mask is not changing over time according to the scene layout changes. The approach employs classifiers, which are trained on some sample images. Although this approach includes a learning phase, the segmentation itself runs with a fixed parameter setting.
“Autonomous Agent Navigation Based on Textural Analysis, Rand Chandler, PhD thesis, University of Florida (2003)” describes a visual architecture for grass segmentation based on texture classification.